BSA Attorney Docket No. BSA 02-22 



TITLE: 



METHOD AND APPARATUS FOR DETECTING, MONITORING, AND QUANTIFYING 
CHANGES IN A VISUAL IMAGE OVER TIME 



Inventors: 

Hugh S. Isaacs 
70 Highland Down 
P.O. Box 62 
Shoreham, New York 11786 

Citizenship : 

U.S.A. 



Alan Shipley 
22 Buckingham Drive 
Sandwich, MA 02563 

Citizenship : 

U.S.A. 



Eric Karplus 

30 Sandwich Road 
East Falmouth, MA 02536 

Citizenship : 



U.S.A. 



This invention was made with Government support under contract number DE- 
AC02-98CH10886, awarded by the U.S. Department of Energy. The Government has 
certain rights in the invention. 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention generally relates to a method and apparatus for locating 
changes that take place over time in a visual image, and relates more particularly to a 
computerized method for digitally processing temporal images to enhance potentially 
minute changes in the image. 
Description of the Prior Art 

It is extremely difficult to detect and monitor small changes that occur in a visual 
image over considerable periods of time due to obvious limitations in human observation, 
perception, and concentration. The average viewer begins to discern change in images 
only after a substantial amount of change has occurred in a relatively concentrated region. 

However, many physical phenomenona, such as the corrosion of a metallic 
surface, typically occur over a relatively broad area and may even regress in some areas 
such that the observer is completely unable to detect any alteration in the surface 
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whatsoever. Thus, there is a need for enhancing changes that occur in a visual image 
over time to enable detection by an average human observer. 

In addition, as with any human observation, a subjective measure of the amount 
of change at any given time is difficult to compare with another such quantity, 
particularly when viewed by different observers. Therefore, there is a need to attach a 
quantitative value that represents changes occurring in a visual image, which may readily 
be stored, processed, and compared with other such quantities. 

Further, conventional techniques for detecting change that utilize, for instance, 
scanning probe microscopes, are severely limited with respect to dynamically locating the 

10 position and degree of changes in an image on a real-time basis. Such approaches are 
typically too cumbersome, especially when presented with a significant amount or rate of 
change over a broad area. Thus, there is a need for a method and apparatus to 
quantitatively detect and monitor changes in visual images in real time. 

Conventional methods exist to digitize images of, for instance, portions of the 
human body. X-ray video imaging manipulates transmitted light and shadows, which 
makes it particularly suitable for studying anatomical objects. However, such techniques 
cannot be used to monitor spectral changes in an image. Furthermore, such techniques 
are not suitable for making quantitative, localized measurements of electrochemical 
activity, which is extremely valuable information for the study of processes such as 

20 corrosion. 

One example of an electrochemical quantity of interest in the study of processes, 
such as corrosion, is pH. Conventional methods have established that it is possible to use 
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spectral information obtained from pH sensitive color indicator dyes to obtain an accurate 
measurement of pH, as described in Robert-Baldo, Gillian L.; Morris, Michael J.; Byrne, 
Robert H., Spectrophotometric Determination of Seawater pH Using Phenol Red, 
Analytical Chemistry, vol. 57 (Nov. 1985) pp. 2564-2577 and Yao, Wensheng; Byrne, 
Robert H., Spectrophotometric Determination of Freshwater pH Using Bromocresol 
Purple and Phenol Red, Environmental Science and Technology, vol 35 no. 6 (Mar. 15, 
2001) pp. 1 197-1201. This art has focused on using a spectrophotometer to monitor 
molecular absorbance of pH sensitive dyes. 

Manufacturers such as OceanOptics located at 380 Main Street, Dimedin, Florida 

10 34698, offer probe-style products that can be used in much the same way as standard 
electrochemical pH probes. A major limitation of both of these techniques, however, is 
that the probes are generally quite large, and even if they could be made very small, the 
fact that the probes only measure a single value averaged over the active area requires 
that a scanning approach be used to obtain a map of pH over an extended surface. This 
would be a very slow process, with the time required increasing in proportion to the 
square of the spatial resolution required, that is, doubling the resolution for an MxN point 
area requires 2Mx2N points. 

OBJECTS AND SUMMARY OF THE INVENTION 
It is an object of the present invention to provide a method and apparatus that are 

20 able to detect, enhance, and quantify physical, chemical, and electrochemical changes 
manifested in a visual image, such as pH, lead content, zinc content, potential difference, 
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pitting, and corrosion, that occur over a period of time so that a human observer can 
readily discern these changes. 

It is another object of the present invention to provide a portable method and 
apparatus that are able to provide rapid, precise quantitative indications of changes that 
occur in a relatively large area over time. 

It is yet another object of the present invention to provide an efficient and cost- 
effective computer-based method and apparatus for detecting and monitoring changes 
that occur in a visual image in near real time with high resolution. 

It is still another object of the present invention to provide a method and apparatus 
1 0 for detecting, monitoring, and quantifying robust spectral information representing 
changes that occur in a visual image over time. 

A method of quantifying measurements associated with a subject using a visual 
image of the subject in accordance with one form of the present invention, which 
incorporates some of the preferred features, includes the steps of acquiring digital 
representations of first and second images of the subject, determining difference 
information in the digital representations of the first and second images, and converting 
the difference information into chemical, physical, electrical, or electrochemical 
information associated with the subject. 

The chemical or physical information includes at least one of pH, lead content, 
20 zinc content, potential difference, or another parameter of chemical, physical, electrical, 
or electrochemical significance. The visual parameter includes at least one of color, tint, 
hue, brightness, and tone. 
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The method may also include the steps of comparing at least a portion of the 
difference information to a threshold value, associating that portion of the difference 
information that is less than or greater than the threshold value with a region of interest, 
and substituting a predetermined value for that portion of the difference information that 
is not within the region of interest. 

Each of the first and second images includes at least one pixel. The pixels 
associated with the first image include a first RGB value, and the pixels associated with 
the second image include a second RGB value. The step of converting difference 
information may also include the steps of converting the first RGB value into a first rgb 
10 tristimulus value; converting the second RGB value into a second rgb tristimulus value; 
converting the first rgb tristimulus value into a first spectral power distribution; 
converting the second rgb tristimulus value into a second spectral power distribution; and • 
obtaining an equation representing the chemical, physical, electrical, or electrochemical 
information as a function of at least one spectral power distribution peak. 

The method also includes subtracting one or more elements of the first spectral 
power distribution and the second spectral power distribution to yield a difference 
spectral power distribution; and multiplying the difference spectral power distribution by 
a derivative of the equation representing the chemical, physical, electrical, or 
electrochemical information as a function of the spectral power distribution peak to 
20 represent a change associated with the subject. 

An apparatus for quantifying measurements associated with a subject using a 
visual image of the subject formed in accordance with one form of the present invention. 



-6- 

which incorporates some of the preferred features, includes a digital camera and a 
computer. The digital camera acquires a digital representation of first and second images 
of the subject. A visual indicator is added to the subject and changes at least one visual 
parameter in response to a chemical, physical, electrical, or electrochemical change 
associated with the subject. 

The computer is responsive to the digital representations and determines 
difference information between the first and second images. The computer converts the 
difference information into information representing the change associated with the 
subject. 

10 These and other objects, features, and advantages of the invention will become 

apparent from the following detailed description of illustrative embodiments thereof, 
which is to be read in connection with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a pictorial representation of an apparatus formed in accordance with 
the present invention. 

Figure 2 is a block diagram of a potentiostat for use in the apparatus formed in 
accordance with the present invention. 

Figures 3a and 3b are flowcharts of a method in accordance with the present 
invention. 

20 Figures 4a-4f are representative computer screens generated by a software 

program in accordance with the present invention. 
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Figure 5 is a flowchart of a preferred algorithm for implementing a threshold 
function in accordance with the present invention. 

Figures 6a-6c are video images of a sample taken at 0, 9, and 19 minutes, 
respectively. 

Figures 7a-7c are pH line profiles corresponding to Figures 6a-6c, respectively. 
Figure 8a is a plot of a color matching function. 

Figure 8b is a plot of a Spectral Power Distribution (SPD) for the RGB value 
(106,137,73). 

Figure 9 is a curve fit for pH as a function of SPD peak product at 450 and 

10 600nm. 

Figures 10a and 10b are plots of pH as a function of measured color value in RGB 
space and XYZ space, respectively. 

Figure 1 la is a pH line profile as a function of time measure with a tungsten 
microelectrode. 

Figure 1 lb is a pH line profile as a function of time measure by the spectral 
method in accordance with the present invention, in which pH is determined from RGB 
color values of a source image. 

Figure 12 is a plot of pH values at selected points as a function of time measured 
with the tungsten microelectrode and the spectral method in accordance with the present 
20 invention. 
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Figures 13a- 13b are time sequence video images of a sample taken at 0, 9, and 19 
minutes, respectively, showing pH maps calculated from RGB values in the source . 
image. 

Figures 14a- 14c are time sequence video images of a sample taken at 0, 9, and 19 
minutes, respectively, showing enhanced difference images referenced to the image in 
Figure 13a. 

Figures 15a and 15b are time sequence video images of a sample taken at 0 and 
19 minutes, respectively, showing changes in pH along an 8mm line with respect to a 
reference condition, measured using three different methods. 

10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

An apparatus 10 for detecting, monitoring, and quantifying changes in a visual 
image over time is shown in Figure 1. The apparatus 10 preferably includes a light 
source 12, which is mounted with a camera 14, such as a digital camera, on a camera 
positioning bracket 16. The camera positioning bracket 16 is preferably mounted on a 
horizontal surface, such as a table 18, and permits the camera 14 and light source 12 to be 
positioned in at least one of an x, y, and z direction above a subject 20 to be monitored. 

The light source 12 is preferably selected to optimize the quality of spectral 
information needed for the desired measurements, and thus may be polychromatic or 
monochromatic. For example, if a single spectral peak is to be monitored, then a 

20 monochromatic source that emits at that spectral peak is preferably used. If multiple 
peaks must be monitored, then a polychromatic source that emits well at each of the 
multiple peaks is preferably used. 
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The subject 20 is preferably mounted on a subject positioning bracket 22, which 
enables the subject 20 to be selectively positioned in at least one of the x, y, and z 
directions. The subject positioning bracket 22 is preferably mounted to the same 
horizontal surface 18 as the camera positioning bracket 16. 

The light source 12 preferably provides an adjustable quantity and direction of 
light and is powered by a supply 24. The camera 14 preferably outputs a digital 
representation of an image of the subject 20 to a computer 26, such as a personal 
computer. The computer 26 is preferably coupled to a display 28 and a keyboard 30 so 
that the user is able to interface with the method and apparatus formed in accordance with 
1 0 the present invention. 

The camera 14 preferably provides a digital representation of an intensity 
corresponding to each of a red, green, and blue color plane for each pixel of an image of 
the subject 20. Stated differently, for each pixel of the image, three digital values are 
provided. Each of these digital values preferably represents the intensity of the red, 
green, or blue color plane corresponding to the associated pixel. Therefore, the camera 
provides robust spectral information concerning each portion of the image so that 
changes occurring in that image may be accurately detected, monitored, and quantified 
over time. 

Figure 1 shows a potentiostat 32, which is shown in greater detail in Figure 2, that 
20 is responsive to electrochemical changes occurring on the subject 20. As shown in 
Figure 2, the potentiostat 32 is an electronic device that controls a voltage difference 
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between a working electrode 34 and a reference electrode 36. Both electrodes are 
preferably incorporated in an electrochemical cell on or in the subject 20. 

The potentiostat 32 preferably implements voltage control by injecting current 
from an auxiliary electrode 38. Typically, the potentiostat 32 measures a current flow 
between the working electrode 34 and the auxiliary electrode 38. The variable that is 
being controlled by the potentiostat 32 is preferably a cell potential and the variable that 
is being measured is preferably a cell current. 

For instance, in corrosion testing, the working electrode 34 is typically coupled to 
corroding metal. The working electrode 34 is preferably not the metallic structure being 
1 0 studied, but rather a small sample used to represent that structure. Generally, the 
electrochemical reaction being studied occurs at the working electrode 34. 

The reference electrode 36 is preferably used to measure the voltage at the 
working electrode 34. A constant electrochemical voltage is preferably measured at the 
reference electrode 36 where there is no current flow through the reference electrode 36. 

The auxiliary electrode 38 is preferably an inert conductor, such as platinum or 
graphite, which is located near the working electrode 34. Current preferably flows into 
the cell 35 from the working electrode 34 and leaves the cell 35 from the auxiliary 
electrode 38. Each of the electrodes is preferably immersed in an electrolyte or 
electrically conductive solution. 
20 As shown in the block diagram of Figure 2, the potentiostat 32 preferably includes 

an electrometer circuit 40 and a current-to-voltage converter circuit 42. These circuits 
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preferably provide unity gain differential amplification with the output voltage for each 
of these devices being the difference between the two inputs. 

The output of the electrometer circuit 40 and the output of the current-to-voltage 
converter circuit 42 are preferably a voltage signal at a voltage node 50 and a current 
signal at a current node 52, respectively. The voltage node 50 and the current node 52 
are preferably coupled to analog-to-digital converters in the computer 26 shown in Figure 
1 for digitization and further processing. 

The electrometer circuit 40 preferably measures the voltage difference between 
the reference electrode 36 and the working electrode 34. The output of the electrometer 
1 0 circuit 40 is preferably used as both a feedback signal 37 in the potentiostat 32 and the 
voltage signal measured at the voltage node 50. An ideal electrometer circuit 40 
preferably has zero input current and infinite input impedance. Thus, current flowing 
through the reference electrode 36 is able to change the potential in the electrometer 
circuit 40. However, since the electrometer circuit 40 preferably has an input current 
near zero, this effect may be ignored. 

The current-to-voltage converter circuit 42 preferably measures the 
electrochemical cell current by forcing the cell current to flow through a current 
measurement resistor 44. The current drop across resistor 44 is preferably a measure of 
the electrochemical cell current. Since cell current in, for instance, a corrosion 
20 experiment often varies by as much as seven orders of magnitude, the electrochemical 
cell current cannot be measured using a single resistor. Therefore, a bank of different 
resistors are preferably switched into the current-to-voltage converter 42 under computer 
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control. This enables a widely varying current to be measured using the appropriate 
value of resistor. 

A control amplifier 46 preferably compares the measured electrochemical cell 
voltage at the voltage node 50 with the desired voltage from an input signal circuit 48 and 
drives current into the electrochemical cell 35 to force these voltages to be the same. 
Since the measured voltage is preferably coupled to a negative input of the control 
amplifier 46, a positive movement in the measured voltage generates a negative output at 
the control amplifier, which counteracts the positive movement in the measured voltage. 

Under normal conditions, the electrochemical cell voltage is preferably controlled 
10 to be identical to the voltage provided by the input signal circuit 48. The input signal 
circuit 48 is preferably a computer controlled voltage source, which is generally 
implemented as the output of a digital-to-analog converter that converts computer 
generated numbers into voltages representing, for instance, constant, ramp, and sinusoidal 
voltage signals. 

Figures 3a and 3b are flowcharts of a method for detecting, monitoring, and 
quantifying changes in a visual image, such as corrosion of a metallic surface in an 
aqueous solution, in accordance with the present invention. In step 54, it is determined 
whether a backgroimd spectral image is to be updated and, if so, a new background 
spectral image is acquired in step 56 and stored in step 58. The background spectral 
20 image is used herein as a reference or baseline image, with which subsequent or source 
images are compared. 
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If a new background spectral image is not required in step 54, the source spectral 
image is acquired in step 60 and stored in step 62. The source spectral image is used 
herein to refer to any image obtained subsequent in time to the background spectral 
image. Analog information is then preferably acquired from the potentiostat and stored 
in step 64. 

It should be noted that background spectral image information, source spectral 
image information, and analog information are preferably stored in digital format, such as 
8, 16, or 32-bit, signed integer, unsigned integer, or floating point formats. The 
background spectral image information and the source spectral image information are 

10 preferably obtained and stored in an unsigned integer format and preferably converted to 
floating point format prior to mathematically manipulating these quantities. 

In step 66, the background spectral image information is preferably subtracted 
from the source spectral image information to obtain difference spectral information. For 
example, assume that the notation "Pbig" refers to a digital representation of the intensity 
of the green (g) color plane corresponding to the first (1) pixel (P) in the background (b) 
image; "Ps2b" refers to the digital representation of the intensity of the blue (b) color 
plane corresponding to the second (2) pixel (P) in the source (s) image; and "Pdsr" refers 
to the digital representation of the intensity of the red (r) color plane corresponding to the 
third (3) pixel (P) in the difference (d) image. 

20 Specifically, Equations (1) - (15) represent a preferred sequence for subtracting 

the background and source spectral image information to generate the difference spectral 
information for pixels 1-5 in step 66, as follows: 
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Pdib = Psib-Pbib 


(1); 


Pdig = Psig-Pbig 


(2); 


Pdir = Ps,r-Pbir 


(3); 


Pd2b = PS2b - Pb2b 


(4); 


Pd2g = PS2g - Pb2g 


(5); 


Pd2r = PS2r-Pb2r 


(6); 


Pd3b = PS3b-Pb3b 


(7); 


Pd3g = PS3g-Pb3g 


(8); 


Pd3r = PS3r-Pb3r 


(9); 


Pd4b = PS4b - Pb4b 


(10); 


Pd4g = PS4g - Pb4g 


(11); 


Pd4r = PS4r " Pb4r 


(12); 


Pd5b = PS5b-Pb5b 


(13); 


Pd5g = PS5g-Pb5g 


(14); and 


Pd5r = PS5r-Pb5r 


(15). 


The preferred difference information for a region of interest is the array of 



difference information (Pd) in each of the color planes for each of the N pixels in the 
region of interest. This difference information may be evaluated in many ways to yield 
meaningful information, such as by selecting the peak histogram value, as described 
20 herein. Alternatively, an algorithm for generating the difference spectral information 
sum may be represented by Equation (16), as follows: 

Pc/ = Z I \Ps,,-Pb,,\ (16); 

1=1 c-b^g^r 
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where Pd is the difference spectral information, i is an index to the pixel number, N is the 
total number of pixels in the region of interest, c is an index to the color plane, b is the 
blue color plane, g is the green color plane, r is the red color plane, Psic is a digital 
representation of the intensity of the color plane denoted by c for the pixel number 
denoted by i corresponding to the source image, and Pbic is a digital representation of the 
intensity of the color plane denoted by c for the pixel number denoted by i corresponding 
to the background image. 

Referring again to Figure 3a, if it is determined that an offset is desired in step 70, 
the offset is added to the difference spectral information in step 72. The offset is used 
10 herein to describe that quantity that when added to the difference spectral information 
makes the resulting image more readily discemable to the user when displayed, such as 
by making the zero difference value appear as a grey color (R,G,B = 128,128,128). It is 
preferable to apply the same offset to each of the R,G,B planes of the difference image, 
but different offset values can also be used for each plane. 

Specifically, Equations (17) - (31) represent a preferred sequence for adding an 
offset to the difference spectral information for pixels 1-5 in step 70, as follows: 



Pdib 


= Pdib + Ob 


(17); 


Pdig 


= Pdig + Og 


(18); 


Pdlr 


= Pdlr + Or 


(19); 


Pd2b 


= Pd2b + Ob 


(20); 


Pd2g 


= Pd2g + 0g 


(21); 


Pd2r 


= Pd2r + 0r 


(22); 
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Pd3b = Pcl3b + Ob 


(23); 


Pd3g = Pd3g + Og 


(24); 


Pd3r = Pd3r + Or 


(25); 


Pd4b = Pd4b + Ob 


(26); 


Pd4g = Pd4g + Og 


(27); 


Pd4r = Pd4r + 0, 


(28); 


Pd5b = Pd5b + Ob 


(29); 


Pd5g = Pd5g + Og 


(30); and 


Pd5r = Pd5r+Or 


(31); 



10 where Ob represents the offset to be added to the blue color plane, Og represents the offset 
to be added to the green color plane, and Or represents the offset to be added to the red 
color plane. 

An algorithm for adding the offset to the difference spectral information sum may 
be represented by Equation (32), as follows: 

Pd = i (32); 

1=1 c=b,g,r 

where Pd is the difference spectral information, i is an index to the pixel number, N is the 
total number of pixels in the region of interest, c is an index to the color plane, b is the 
blue color plane, g is the green color plane, r is the red color plane, Pdic is a digital 
representation of the intensity of the color plane denoted by c for the pixel number 
20 denoted by i corresponding to the difference image, and Oc is the offset value 
corresponding to the color plane denoted by c. 
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If it is determined that gain is required in step 72, the difference spectral 
information is preferably multiplied by the desired gain factor in step 74. If the offset has 
been chosen not to be added to the difference spectral information in step 68, the process 
circumvents step 70 and proceeds to step 72. Likewise, if gain is not desired in step 72, 
the process circumvents step 74 and proceeds to step 76, as shown in Figure 3b. The gain 
factor is used herein to amplify the difference spectral information so that it can more 
readily be viewed by the user on a display. 

Specifically, Equations (33) - (47) represent a preferred sequence for multiplying 
the difference spectral information by the gain factor for pixels 1-5 in step 76, as follows: 



Pdib = Pdib-Gb 


(33); 


Pd,g = Pd,g'Gg 


(34); 


Pd,r = Pd,r-Gr 


(35); 


Pd2b = Pd2b-Gb 


(36); 


Pd2g = Pd2g-Gg 


(37); 


Pd2r = Pd2r'Gr 


(38); 


Pdjb = Pdsb • Gb 


(39); 


Pd3g = Pd3g'Gg 


(40); 


Pd3r = Pd3r-Gr 


(41); 


Pd4b = Pd4b • Gb 


(42); 


Pd4g = Pd4g • Gg 


(43); 


Pd4r = Pd4r • Gf 


(44); 


Pd5b = Pd5b-Gb 


(45); 
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Pdsg = Pdsg • Gg (46); and 

Pd5r = Pd5r-Gr (47); 

where Gb represents the offset to be added to the blue color plane, Gg represents the offset 
to be added to the green color plane, and Gr represents the offset to be added to the red 
color plane. 

An algorithm for multiplying the difference spectral information sum by the gain 
factor may be represented by Equation (48), as follows: 

Pd = i Z \Pd^*Gl (48), 

i=l c=b,g,r 

where Pd is the difference spectral information, i is an index for the pixel number, N is a 
10 total number of pixels in the region of interest, c is an index to the color plane, b is the 
blue color plane, g is the green color plane, r is the red color plane, Pdic is a digital 
representation of the intensity of the color plane denoted by c for the pixel number 
denoted by i corresponding to the difference image, and Gc is the gain factor 
corresponding to the color plane denoted by c. 

If a difference threshold is desired in step 78, a threshold function is preferably 
applied to the difference spectral information in step 80. The threshold function 
essentially compares difference spectral information associated with each of the red, 
green, and blue planes for each pixel to one or more desired threshold values or ranges. 
If the spectral information is less than or greater than the threshold or within a threshold 
20 range, this particular portion of the difference spectral information is preferably not 
shown on the display or assigned a default or predetermined value. Additional details 
concerning the threshold function are shown in Figure 5, which is described below. 
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In step 82 of Figure Sb, the process preferably generates a histogram representing 
pixel intensity as a function of the quantity of pixels having a particular intensity for each 
of the red, green, and blue color planes. If the threshold function has been applied, the 
histogram is preferably limited to a region of interest determined by the desired threshold 
value or range. A graph is displayed to the user in step 84 with pixel intensity as the y- 
axis and the quantity of pixels having a particular intensity as the x-axis. The pixel 
information for each of the red, green, and blue color planes is preferably displayed as a 
red line, a green line, and a blue line, respectively, on the graph. 

Peaks in the histogram are then preferably determined for each of the red, green, 
10 and blue color planes in step 86. The peak information is preferably displayed as an 
additional graph having pixel intensity as the y-axis and time as the x-axis with red, 
green, and blue lines representing each of the red, green, and blue color planes in step 88. 

In step 90, the background spectral image and a processed spectral image are 
preferably displayed to the user. The processed spectral image is preferably the 
difference spectral information as optionally modified by the offset, gain, and threshold, 
values. The processed spectral image may also result from overlaying or superimposing 
the difference spectral information on one or more background spectral images. 

The analog information obtained from the potentiostat in step 64 is preferably 
displayed in graphical form in step 92. The analog information is preferably displayed as 
20 a graph of voltage or current on the y-axis and the sample number on the x-axis. Since 
samples of the analog information are preferably obtained consecutively over time, the 
analog information is essentially displayed as a function of time. The process then 
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preferably determines whether a halt is indicated in step 94 and either returns to step 54 
to re-execute the process or ends in step 96. 

Figures 4a - 4f show representative computer displays that enable the user to 
interface with the process in accordance with the present invention. Figure 4a shows the 
histogram 98 of pixel intensity as a function of a quantity of pixels having a particular 
intensity. As described above, the histogram preferably includes a red line 100, blue line 
102, and a green line 104 that represent digital information corresponding to each of the 
red, blue, and green color planes, respectively. 

Cursors 106, 108 provide x and y coordinates corresponding to their location on 
10 the histogram so that the user can assign specific values to selected portions of the 

histogram. The x and y values for each of the cursors 106, 108 is preferably provided in 
a display block 1 10. For instance, as shown in Figure 4a, cursor 108 has an x value of 
1 19.66 and a y value of 27.72, as indicated in block 1 10. 

Figure 4a also preferably shows a graph of histogram peak intensities 1 12 as a 
function of time for each of the red, green, and blue color planes. Thus, as with the 
histogram 98, the histogram peak graph 112 preferably includes three lines, each of 
which represents one of the red, green, and blue color planes. 

The lower portion of the display is preferably used to input user-defined 
parameters. For instance, the user is preferably able to select the desired offset in field 
20 114 and the desired gain factor or multiplier in field 116. The offset and gain factors 
preferably correspond to one or more of the red, green, and blue color planes, which are 
selected in field 118. 
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The user is preferably able to save the processed image by selecting field 120. 
The parameters discussed above are preferably included within a difference parameter 
field 122 , which is one of a plurality of tab selectable fields within the same general field 
on the display. 

In an acquisition parameter field 124, the user-selectable fields include the 
filename to which the saved images are assigned and the number of spectral images that 
are to be obtained before acquiring a new backgroxmd image. In addition, the user is 
preferably able to select the number of source images and background images over which 
the corresponding spectral information is averaged, as well as the length of an update 

10 interval, which is the interval between acquisitions of different source images. 

The acquisition parameters are preferably saved by selecting a save sequence field 
126 and processing is initiated by selecting a process field 128. A new backgroxmd 
spectral image is preferably acquired in response to selecting a background field 130 and 
analog information firom the potentiostat is preferably obtained in response to selecting a 
record A/D field 132. A new background image is preferably acquired by selecting the 
background field 131 and the displayed images are focused by selecting a focus field 133. 
Processing is preferably paused or stopped by selecting fields 134 and 136, respectively. 
Additional hardware settings, such as buffer sizes and camera specifications may be 
accessed through a hardware settings field 138. 

20 In the bottom rightmost portion of the display, the user is preferably able to select 

one or more images to be displayed, that is, one of the background, processed, or source 
images, by selecting the desired image in a view field 140. The user is also preferably 
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able to show or hide the image or source display windows by selecting fields 141 or 143, 
respectively. The user is also preferably able to turn the analog data acquisition on or off 
by selecting field 145 and to show or hide window tools for defining a region of interest 
by selecting field 147. 

Figure 4b is substantially similar to Figure 4a, except that a threshold field 148 
has been selected rather than the difference parameters field 122 shown in Figure 4a. The 
threshold field 1 48 preferably enables the iiser to select threshold ranges for each of the 
red, green, and blue color planes, by selecting the appropriate values in fields 150, 152, 
154, respectively. A threshold mode is selected in field 156, which preferably determines 
10 those color planes subject to the threshold function. A value to be substituted for those 
portions of the image that are outside of the threshold range is preferably provided by the 
user in field 158. 

Figure 4c is substantially similar to Figure 4a, except that an equalize field 160 
has been chosen rather than the difference parameters field 122. By selecting an equalize 
field 162, the user is essentially able to spread the spectral information over a broader 
region along the x-axis on the display to enable finer details to be more readily 
discemable by the user. 

Figure 4d is substantially similar to Figure 4a, except that a pixel intensity field is 
shown rather than the difference parameters field 122 shown in Figure 4a. The user 
20 preferably selects, in field 166, a range of pixel intensity for displaying, as well as a 
center of the displayed pixel intensity at field 168. Alternately, the user may select an 
auto scale field 170, which essentially formats the displayed pixel intensity to most 
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efficiently use the entire display field in accordance with the spectral information in the 
queue. Pixel intensity is preferable saved by selecting field 172. 

By selecting the analog-to-digital field 145 shown in Figure 4a, a mean value of 
the analog information provided by the potentiostat is preferably displayed to the user on 
an analog display field 190 shown in Figure 4e, A graphical display 176 of voltage as a 
function of sample number or time is displayed with at least two lines having different 
colors representing plot 0 and plot 1. The differently colored plots preferably represent 
different signals used for excitation of the potentiostat, such as triangular or sinusoidal 
signals, or the current response of the potentiostat. 

10 Cursors are preferably provided on the graphical display 176, and the x and y 

coordinates associated with these cursors are provided in field 178. Alternatively, a 
graph of current as a fimction of sample number or time may be displayed in field 176 by 
selecting an alternative value in field 180. Information concerning a particular image 
number, as selected in field 182, and a channel number, as selected in field 184, is 
preferably provided in field 186, such as the average value and standard deviation for the 
analog information shown in field 188. 

Figure 4f is substantially similar to Figure 4e, except that a standard deviation 
field 192 is shown rather than the mean field 190. The graph 194 preferably shows the 
standard deviation of voltage as a function of sample number or time for one or more 

20 excitation signals or plots, which are preferably represented in different colors on the 
graph 194. 
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Figure 5 is a flowchart that provides additional details concerning the threshold 
function performed in step 80 in Figure 3b. In step 196, which follows step 78 shown in 
Figure 3a, the index I, which denotes the current pixel number, is preferably initiaHzed to 
one and the index c, which denotes the current color plane, is preferably initialized to 
blue. The digital representation for the difference spectral information Pdic associated 
with the current pixel number and current color plane, is then preferably compared to an 
upper threshold Tuc and a lower threshold TU associated with the current color plane in 
step 198. 

If the difference spectral information is not within the threshold range, the 
10 difference spectral information is preferably overwritten with a default or predetermined 
value representing, for instance, the intensity of a blank background in step 200. If the 
difference spectral information is within the threshold range, index i is preferably 
incremented in step 202. If index i is not greater than the total number of pixels N in step 
204, the routine preferably returns to step 198. 

If index i is greater than N, index c is not currently green in step 206, and index c 
in not currently red in step 210, index i is preferably initialized to 1, index c is set to 
green in step 208, and the routine returns to step 198. If index c is determined to be green 
in step 206, then index i is preferably initialized to 1, index c is set to red in step 212, and 
the routine returns to step 198. If index c was red in step 210 the routine continues with 
20 step 82, as shown in Figure 3b. 

A pH sensitive color indicator is preferably placed in a solution that is on top of a 
sample to be studied. In some cases, it is desirable to include a gelling agent in the 
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solution to slow down bulk transport so that larger gradients may be built up. Alternative 
types of indicator dyes may be used to monitor other electrochemically significant 
quantities. For example, there are a wide variety of chemical spot tests (see Andrew 
Holmes, Rapid Spot Testing of Metals, Alloys and Coatings, Metal Finishing Information 

Services Ltd. and ASM International, pp. (2002) which is incorporated herein by 

reference) that can be used to test for the presence of specific chemicals in a solution, 
such as lead, iron, and hydrogen ions (pH). In addition, fluorescent indicators exist that 
respond to metal ions, such as zinc, and fluorescent indicators exist that respond to a 
potential difference across a membrane. 

10 If a polychromatic light source, such as a tungsten-halogen bulb, is used to 

illuminate the sample, the reflected light preferably includes spectral information 
regarding the molecular absorbance of the indicator dye in the solution. Spectrally 
filtered light is used to illuminate the sample with fluorescent markers and the spectral 
information from the fluorescent emissions may be monitored. In either case, a color 
image digitizer is preferably used to capture spectral information firom the resulting light. 

Monochromatic light is preferably used to measure spectral absorbance or 
fluorescence information at a fixed wavelength when absorption or fluorescence at only 
one wavelength is necessary for spectral quantification. If monochromatic illumination is 
used, then a monochromatic image digitizer may be used. In the case of fluorescence 

20 detection, it is generally preferably to use an interference filter in fi-ont of the image 
digitizer to block the illumination light and pass the fluorescence emission. 



-26- 

Correlation between the measured spectral information and an electrochemically 
significant measurement is important. The following example demonstrates application 
of this technique with a pH indicator dye having a pH range of 4-10 placed in a solution 
over a metal sample. A gelling agent is used to slow down bulk transport so that larger 
gradients may be observed for longer periods of time. The sample is illuminated from 
above with a tungsten-halogen lamp without the use of spectral filters. A color video 
camera with a low magnification lens is preferably used to acquire an RGB encoded 
image. 

A time sequence of three images is shown in Figures 6a-6c taken at 0, 9, and 19 
10 minutes, respectively. To the right of each image is a corresponding graph, Figures 7a- 
7c, showing the pH profile taken along an 8mm line measured with a tungsten 
microelectrode, as indicated by a dashed line 214, and the spectral method described 
below, as indicated by a solid line 216. 

A white star-burst 218 in the lower left of the first two images. Figures 6a and 6b, 
represents a tungsten microelectrode immersed in the gelled solution to measure the 
localized pH based on the electrochemical potential of the electrode tip with respect to a 
stable reference electrode. The microelectrode is calibrated in a known buffer solution, 
and stepped at regular intervals of 0.1 16 mm across the gelled solution to estimate the pH 
gradient shown by the dashed lines 214 in Figures 7a-7c. The path along which pH is 
20 measured by the microelectrode is shown as a white line 220 in each of Figures 6a-6c. 
The time required to scan an 8mm path with this resolution is about 400 seconds. 
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The spectral content of the image data along the probe path is preferably 
calibrated from the pH values obtained by the tungsten microelectrode using the 
procedure in accordance with the present invention. The pH values along the probe path 
are then preferably calculated from the spectral information in the image along this path 
and plotted as a solid line in the charts shown in Figures 7a-7c. The spectral values are 
preferably obtained from RGB values along a single pixel line, and the RGB values in 
this line are smoothed by averaging the value for each pixel with the value of its nearest 
neighbor pixels to the right and left. 

Each pixel represents a physical dimension of 0.057 mm on the sample. The time 
10 required to acquire pixel data for the full image is less than 17 milliseconds. The data 
from the line along the probe path, which is analyzed to generate the charts in Figures 7a- 
7c, may be acquired in less than 10 microseconds (140 pixels at a 20 MHz clock rate). 
This is greater than seven orders of magnitude faster than the acquisition rate using the 
microelectrode. 

The spectral content of the reflected light is preferably detected by the color 
camera and determined as follows: 



1. Each pixel in the image is preferably digitized to obtain an RGB value; 



2. The RGB value is preferably converted to an rgb tristimulus value, as 



follows: 



20 



r = R/(R+G+B) 



(49) 



g = G/(R+G+B) 



(50) 



b = B/(R+G+B) 



(51). 



-28- 

For example, the digitized color value RGB = (106,137,73) becomes the 
tristimulus value rgb = (0.335,0.434,0.231). The tristimulus value normalizes the color 
content of the digitized value against intensity to facilitate color matching calculations. 

3. The rgb tristimulus value is preferably converted to a Spectral Power 
Distribution SPD using the CIE 1931 color matching functions r, g, b described 
in Wyszecki, Gunther, and Stiles, Color Science: Concepts and Methods, 
Quantitative Data and Formulae, Second Edition, John Wiley & Sons, Inc., New 
York, 2000, pp.750-751, the relevant portions of which are incorporated herein by 
reference: 

10 SPD = rr + gg + bb (52). 

The rgb color matching functions are shown in Figure 8a and an example of the 
SPD for the RGB value (106,137,73) is shown in Figure 8b. The SPD calculated in this 
maimer is technically a metamer, which means that the human eye will perceive it to be 
the same color as the light from the sample, while, in fact, the light from the sample may 
actually have a different SPD. As will be seen from the following results, this does not 
prevent a meaningful determination of pH based on the calculated SPD. 

4. The SPD for each pixel in the path of the tungsten microelectrode is 
calculated and transmission peaks are observed near 450 nm and 600 nm in each 
SPD throughout the pH range of --7.6 to ---4.6 along the tungsten microelectrode 

20 path. This agrees with the expectation that high pH values produce a blue color 

and low pH values produce a red color. pH may be accurately estimated based on 
calibrations involving absorption peaks rather than transmission peaks. In the 
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present invention, a multivariable function is preferably used to map RGB color 
values to a pH value. 

In one embodiment of the present invention, the product of the SPD transmission 
peaks calculated, as described above, at two wavelengths provides a parameter that is 
preferably used to fit a curve for calculating pH. 

In the case shown in Figure 9, which shows a curve fit for pH as a function of 
SPD peak product at 450 and 600 nm, the SPD peak product x is preferably calculated 
using the following equation: 

x = SPD450 *SPD6oo (53), 
10 and the curve fit for calculating pH from the SPD peak product x is expressed as the 
following: 

pH = -8.531E-lx^ + 1.503E+3x^ + -1.013E+5x + 2.390E+6 (54). 

The SPD peak product is not necessarily the preferred parameter to use for 
calculating pH since this parameter may not perform well above a pH of 7.5 or below a 
pH of 5.5. The performance of this parameter at these extremes is due in part to the 
intrinsic spectral characteristics of the illumination source, camera, sample, and solution. 

For example, the RGB values used to generate the SPD peak product for pH 
values above 7.3 were near the noise level of the camera, which represents very dark 
colors. In addition, it was observed in the data set described above that the red values 
20 from the camera were saturated for most of the pH values below 6, which resulted in a 
loss of dynamic range and accuracy for the SPD peaks. Even under these conditions. 
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close agreement was obtained between the pH values from the tungsten electrode and the 
pH values calculated using the SPD peak product parameter. 

The preferred parameter for calculating pH depends on a number of factors 
including, but not limited to the spectral output of the illumination source, the spectral 
sensitivity of the imaging device, the emissivity of the sample being observed, and the 
spectral transmission characteristics of the indicator and the solution being used. 

Another example of a method for mapping RGB values to pH in accordance with 
the present invention will now be described. Measurement of RGB values obtained with 
a video camera are transformed into rgb tristimulus values and plotted as a function of 

10 pH, as measured with the tungsten microelectrode. Alternatively, the RGB values may 
be transformed into a different color space, for example, the CIE 1931 X,Y,Z space, as 
described in Wyszecki, Gunther, and Stiles, Color Science: Concepts and Methods, 
Quantitative Data and Formulae, Second Edition, John Wiley & Sons, Inc. ,New York, 
2000, p. 139, the relevant portions of which are incorporated herein by reference. 
Examples of these two scenarios are provided in Figures 10a and 10b, which show plots 
of pH as a function of measured color value in RGB space, as shown in Figure 10a, and 
X,Y,Z space, as shown in Figure 10b. 

The color space values, referred to here in general as (a,b,c), define a line in 
three-dimensional space such that evaluation of a function f(a,b,c) produces a pH value, 

20 that is, pH = f(a,b,c). The difficulties of this approach include finding the form of the 
equation f(x) that will fit the multivariable data obtained in the calibration process, and 
defining the range of values for which the fitted curve can be evaluated, that is, what to 
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do if the dependent coordinates are not in the range of values for which the curve is 
defined during calibration. 

It may be preferable to adjust the illumination level and camera shutter speed so 
that none of the pixels are saturated at any point during the acquisition. Once the 
illumination level has been set, the camera is preferably white balanced. These 
precautions ensure that the camera is capable of obtaining the widest dynamic range of 
spectral information during the experiment. 

A comparison of pH gradient data obtained from three subsequent scans of the 
tungsten microelectrode is shown in Figure 1 la and compared with the pH gradient data 
10 along the same path calculated from pixel lines in three images acquired near the start of 
each microelectrode scan. The gradients become steeper with each subsequent 
microelectrode scan, a pattern which can be seen in both pH data measured with the 
tungsten microelectrode in Figure 11a and the pH data measured with the spectral 
information obtained with a color video camera shown in Figure 1 lb. 

Thus, the method formed in accordance with the present invention provides the 
capability of obtaining a quantitative measurement of pH from a digitized image. In 
essence, the method provides a very tightly packed array of pH microelectrodes operating 
at very high speed. It is possible to select a region of interest as small as a single pixel, 
and monitor its pH as a function of time. This permits a highly localized measurement to 
20 be made. 

Figure 12 shows pH values at selected points as a function of time measured with 
the tungsten microelectrode at curves 220 and 222. Data omitted from the curves 224 in 
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Figure 12, which shows the spectral method in which pH is determined froni the source 
image RGB color values, was acquired while the tungsten microelectrode obscured the 
pixel. 

The pH as a function of time for two additional points, which were not along the 
probe path but had a similar initial color, is plotted for comparison. This shows the type 
of data that can be obtained if there is no scanning microelectrode obstructing the image 
digitizer. The pH measurements as a function of time correspond to a roughly 60 x 60 
micron area that is monitored continuously without the use of a microelectrode. 

Since it is possible to analyze each pixel in the image to generate a pH value from 
10 the RGB value, it is possible to transform each pixel in an acquired image into a pH 
value. The result of applying this method to the images in Figures 6a-6c is provided in 
Figures 13a- 13c, which show a time sequence of pH maps calculated from RGB values in 
the source image at 0, 9, and 19 minutes, respectively. 

The white areas represent locations where the SPD Peak Product parameter is out 
of range for calculating the pH value. Figures 14a- 14c show enhanced difference images 
referenced to the image in Figure 14a. A color scale 226 in Figure 13a shows the pH 
color codes used in Figures 13a-13c. 

Each pH value in Figures 14a- 14c requires calculations made with floating point 
buffers to obtain a reasonably accurate value. Each calculation includes 1 add and 3 
20 divides to go from RGB to rgb, 6 multiplies and four adds followed by a multiply to 

produce an SPD peak product value, followed by 9 multiplies and three adds to obtain a 
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pH value. While conventional computers perform these operations quite rapidly, another 
method may be used to more rapidly estimate pH from a difference image. 

The present invention is able to rapidly detect changes in a field of view by 
looking at differences between a current digitized image and a reference digitized image, 
and translate those differences into electrochemically important quantities. The RGB 
difference images v^th respect to the 0 minute image (Figure 13a, multiplier=0.5, 
offset=128) are shown to the right of the pH image in Figure 14a. 

Although the enhanced RGB difference image simplifies the identification of 
regions where there are changes in pH, estimating pH changes requires the determination 
10 of a difference image based on rgb tristimulus values. Equation (53) has the general form 
pH = f(x), and thus an estimate of the change in pH may be obtained as follows: 



For example, if x = SPD450 * SPD600 as in Equation (53), then f (x) is preferably 
obtained from Equation (54) as follows: 



This function is preferably calculated once for the reference image. Subsequent 
changes in pH are preferably determined from the difference image provided Ax can be 
calculated from the RGB values in the difference image. Evaluating Equation (53) for 
the reference image yields the following: 



ApH=^f{x)Ax 



(55). . 



f(x) = -2.5593 + 3.006E+3 jc + -1.013E+5 



(56). 



20 



X = SPD450,ref*SPD600,rcf 



(57). 
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In general, subsequent images preferably have a different Spectral Power 
Distribution peak product, which can be expressed follows: 

jc + Ax = SPD450 * SPDeoo (58). 
Subtracting Equations (57) from (58) yields the following expression for Ax: 

Ax = SPD45O * SPD6OO - SPD450,ref * SPD600,ref (59). 

The values SPD45o,difT and SPDeocdiff from the Spectral Power Distributions calculated 
using the rgb tristimulus values associated with the difference image are as follows: 

SPD450,diff = ( SPD45O - SPD450.ref ) (60) 
SPD600,diff = ( SPD6OO - SPD600.ref ) (61), 

1 0 The method for obtaining tristimulus values for the difference image is to transform the 
source image into tristimulus rgb values using Equations (49), (50), and (51), and to take 
the difference between the tristimulus image and the tristimulus reference image to 
determine the SPD45o,diff and SPD6oo,difif values using Equation (52). Equations (60) and 
(61) can be solved for SPD450 and SPD600 and substituted into Equation (59). The result 
is as follows: 

AX= SPD450,diff * SPD600,ref + SPDfiOO.diff * SPD450,ref+ SPD450,diff * SPD600,diff (62). 

Substituting this value, which requires 3 muhiplies and two adds, into Equation 
(56) yields a value for ApH with one additional multiplication. Thus, the 9 multiplies and 
three adds required to obtain a pH value from the source RGB values may be replaced by 
20 3 subtractions (to get the rgb difference value), 4 multiplies, and two adds to estimate a 
change in pH relative to a reference image, provided the f (xref) values in Equation (57) 
for the reference image are calculated in advance and stored in a floating point buffer. 
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Figures 15a and 15b compare this method of estimating pH changes, as shown by 
a thick solid line 226, with two other methods at 9 and 19 minutes, respectively. The 
profile is taken along the same line shown in the images of Figures 6a-6c. The first 
comparison is made by transforming the source image RGB values into pH units, as 
described above, and then subtracting the pH values for the current image from the pH 
values for the reference image. Since pH is a scalar quantity, the result may be displayed 
as monochromatic images, or they can be color enhanced to facilitate recognition. 

For comparison with the microelectrode method, the result of this method is 
provided in Figures 15a and 15b, which show changes in pH along an 8mm line with 

10 respect to a reference condition measured using 3 different methods, as a thin line 228 
with small rectangular box data points (SPD PP). The pH change values obtained with 
the two different spectral methods are in substantial agreement. While there is more 
noise in the values obtained with the spectral method, in some instances this will be an 
acceptable sacrifice, considering that the spectral method allows measurements to be 
made about seven orders of magnitude faster. 

A second comparison is made using a tungsten microelectrode to perform 
subsequent scans along a line, and calculating the change in pH using the difference in 
the microelectrode measurements, as indicated by the dashed lines 230 in Figures 1 5a and 
15b. This is a slow process, in this case requiring about 10 minutes per scan, and thus the 

20 resuhs of this method are not expected to precisely match the resuhs of the spectral 

method. However, the values in the 8- 10mm range agree more than the values in the 2- 
4mm range since their acquisition times are more similar. 



-36- 

Therefore, the method and apparatus formed in accordance with the present 
invention enable measurement of changes in pH from spectral information in digitized 
images. The general method may be applied to any system where a set of calibration 
values can be established that correlate an electrochemicaliy significant quantity with 
spectral measurements made by means of an imaging detector. The general method is to 
transform the spectral measurements into a single variable x that can be mapped as a 
function f(x) to determine the electrochemicaliy significant quantity. Electrochemicaliy 
significant quantities measured in this way can be used to determine rates and/or amounts 
of electrochemical process. 
10 In some situations, it may be preferable to estimate changes in the 

electrochemicaliy significant quantity from a difference image with respect to a reference 
image. In this case, the change in the electrochemicaliy significant quantity is estimated 
as Af(x) = f (xref) Ax. The benefit of using this approach is that, in some cases, the 
calculation of Ax from the difference of the present image with respect to the reference 
image shown in Figures 14a-c is faster than the calculation of the value of f(x) from the 
present image. 

Therefore, the method and apparatus formed in accordance with the present 
invention are able to detect, enhance, and quantify physical, chemical, and 
electrochemical changes manifested in a visual image, such as pH, lead content, zinc 
20 content, potential difference, pitting, and corrosion, that occur over a period of time so 
that a human observer can readily discern these changes. In addition, the method and 
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apparatus provide an efficient computer-based technique for detecting and monitoring 
changes in a visual image in real time using robust spectral information. 

Although illustrative embodiments of the present invention have been described 
herein with reference to the accompanying drawing, it is to be understood that the 
invention is not limited to those precise embodiments, and that various other changes and 
modifications may be effected therein by one skilled in the art without departing from the 
scope or spirit of the invention. 



